Energy-Efficient Inference Accelerator for Memory-Augmented Neural Networks on an FPGA

Memory-augmented neural networks (MANNs) are designed for question-answering tasks. It is difficult to run a MANN effectively on accelerators designed for other neural networks (NNs), in particular on mobile devices, because MANNs require recurrent data paths and various types of operations related to external memory access. We implement an accelerator for MANNs on a field-programmable gate array (FPGA) based on a data flow architecture. Inference times are also reduced by inference thresholding, which is a data-based maximum inner-product search specialized for natural language tasks. Measurements on the bAbI data show that the energy efficiency of the accelerator (FLOPS/kJ) was higher than that of an NVIDIA TITAN V GPU by a factor of about 125, increasing to 140 with inference thresholdingComment: Accepted to DATE 201

Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection

Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-generation of neural networks due to their event-driven and low-powered nature. SNNs, however, are difficult to train, mainly owing to their complex dynamics of neurons and non-differentiable spike operations. Furthermore, their applications have been limited to relatively simple tasks such as image classification. In this study, we investigate the performance degradation of SNNs in a more challenging regression problem (i.e., object detection). Through our in-depth analysis, we introduce two novel methods: channel-wise normalization and signed neuron with imbalanced threshold, both of which provide fast and accurate information transmission for deep SNNs. Consequently, we present a first spiked-based object detection model, called Spiking-YOLO. Our experiments show that Spiking-YOLO achieves remarkable results that are comparable (up to 98%) to those of Tiny YOLO on non-trivial datasets, PASCAL VOC and MS COCO. Furthermore, Spiking-YOLO on a neuromorphic chip consumes approximately 280 times less energy than Tiny YOLO and converges 2.3 to 4 times faster than previous SNN conversion methods.Comment: Accepted to AAAI 202

SimFLE: Simple Facial Landmark Encoding for Self-Supervised Facial Expression Recognition in the Wild

One of the key issues in facial expression recognition in the wild (FER-W) is that curating large-scale labeled facial images is challenging due to the inherent complexity and ambiguity of facial images. Therefore, in this paper, we propose a self-supervised simple facial landmark encoding (SimFLE) method that can learn effective encoding of facial landmarks, which are important features for improving the performance of FER-W, without expensive labels. Specifically, we introduce novel FaceMAE module for this purpose. FaceMAE reconstructs masked facial images with elaborately designed semantic masking. Unlike previous random masking, semantic masking is conducted based on channel information processed in the backbone, so rich semantics of channels can be explored. Additionally, the semantic masking process is fully trainable, enabling FaceMAE to guide the backbone to learn spatial details and contextual properties of fine-grained facial landmarks. Experimental results on several FER-W benchmarks prove that the proposed SimFLE is superior in facial landmark localization and noticeably improved performance compared to the supervised baseline and other self-supervised methods

Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks

The spiking neural networks (SNNs) are considered as one of the most promising artificial neural networks due to their energy efficient computing capability. Recently, conversion of a trained deep neural network to an SNN has improved the accuracy of deep SNNs. However, most of the previous studies have not achieved satisfactory results in terms of inference speed and energy efficiency. In this paper, we propose a fast and energy-efficient information transmission method with burst spikes and hybrid neural coding scheme in deep SNNs. Our experimental results showed the proposed methods can improve inference energy efficiency and shorten the latency.Comment: Accepted to DAC 201

Energy-efficient Knowledge Distillation for Spiking Neural Networks

Spiking neural networks (SNNs) have been gaining interest as energy-efficient alternatives of conventional artificial neural networks (ANNs) due to their event-driven computation. Considering the future deployment of SNN models to constrained neuromorphic devices, many studies have applied techniques originally used for ANN model compression, such as network quantization, pruning, and knowledge distillation, to SNNs. Among them, existing works on knowledge distillation reported accuracy improvements of student SNN model. However, analysis on energy efficiency, which is also an important feature of SNN, was absent. In this paper, we thoroughly analyze the performance of the distilled SNN model in terms of accuracy and energy efficiency. In the process, we observe a substantial increase in the number of spikes, leading to energy inefficiency, when using the conventional knowledge distillation methods. Based on this analysis, to achieve energy efficiency, we propose a novel knowledge distillation method with heterogeneous temperature parameters. We evaluate our method on two different datasets and show that the resulting SNN student satisfies both accuracy improvement and reduction of the number of spikes. On MNIST dataset, our proposed student SNN achieves up to 0.09% higher accuracy and produces 65% less spikes compared to the student SNN trained with conventional knowledge distillation method. We also compare the results with other SNN compression techniques and training methods

Removal of Alpha-Gal Epitopes from Porcine Aortic Valve and Pericardium using Recombinant Human Alpha Galactosidase A

It has been reported that the immune response due to α-Gal epitopes is an important factor in tissue valve failure. The elimination of the interaction between the natural anti-Gal antibodies and α-gal epitopes on the xenografts is a prerequisite to the success of xenografts in humans. Previously, we reported that the green coffee bean α-galactosidase could remove all α-Gal epitopes from cell surface of porcine aortic valve and pericardial tissue, but it has limitations on cost effectiveness. In this study we wanted to know whether the recently produced recombinant human α-galactosidase A has the same effective enzymatic activity as green coffee bean α-galactosidase in removing α-Gal epitopes from the same tissues. After treating fresh porcine aortic valve and pericardial tissue with recombinant α-galactosidase A, each sample was stained with Griffonia simplicifolia type I isolectin B4 indirect immunoperoxidase avidin-biotin technique. We then examined whether the α-Gal epitopes were reduced or abolished in each consecutive concentration of recombinant α-galactosidase A by comparing the degree of the Griffonia simplicifolia isolectin B4 staining. As a result, the recombinant α-galactosidase A could remove cell surface α-Gals on porcine aortic valve and pericardial tissue as effectively as green coffee bean α-galactosidase

Surgical Treatment of Native Valve Aspergillus Endocarditis and Fungemic Vascular Complications

Systemic infection with Aspergillus is an opportunistic disease that affects mainly immunocompromised hosts, and is associated with a high mortality rate. It typically occurs in patients with several predisposing factors, but Aspergillus endocarditis of native valves is rare and experience in diagnosis and treatment is limited. We report a case of native valve endocarditis caused by Aspergillus. A 35-yr-old male patient who underwent pericardiocentesis four months previously for pericardial effusion of unknown etiology presented with right leg pain and absence of the right femoral artery pulse. Cardiac echocardiography revealed severe mitral insufficiency with large mobile vegetations, and computed tomographic angiography showed embolic occlusion of both common iliac arteries. We performed mitral valve replacement and thromoembolectomy, and Aspergillus was identified as the vegetation. We started intravenous amphotericin B and oral itraconazole, but systemic complications developed including superior mesenteric artery aneurysm and gastrointestinal bleeding. After aggressive management, the patient was discharged 78 days post surgery on oral itraconazole. He was well at 12 months post discharge but died in a traffic accident 13 months after discharge

Dual Pointer Network for Fast Extraction of Multiple Relations in a Sentence

Relation extraction is a type of information extraction task that recognizes semantic relationships between entities in a sentence. Many previous studies have focused on extracting only one semantic relation between two entities in a single sentence. However, multiple entities in a sentence are associated through various relations. To address this issue, we proposed a relation extraction model based on a dual pointer network with a multi-head attention mechanism. The proposed model finds n-to-1 subject–object relations using a forward object decoder. Then, it finds 1-to-n subject–object relations using a backward subject decoder. Our experiments confirmed that the proposed model outperformed previous models, with an F1-score of 80.8% for the ACE (automatic content extraction) 2005 corpus and an F1-score of 78.3% for the NYT (New York Times) corpus